Skip to content

fix(llm): cap provider response bodies#56

Open
CryptoJones wants to merge 1 commit into
mainfrom
fix/48-llm-response-cap
Open

fix(llm): cap provider response bodies#56
CryptoJones wants to merge 1 commit into
mainfrom
fix/48-llm-response-cap

Conversation

@CryptoJones

Copy link
Copy Markdown
Owner

Summary

  • Send LLM requests with HttpCompletionOption.ResponseHeadersRead.
  • Reject declared oversized LLM success responses before reading the body.
  • Stream success and error response bodies with hard byte caps.
  • Preserve concise logging for oversized upstream error bodies without allocating the full payload.
  • Add isolated tests for normal completions, declared oversized success responses, streamed oversized success responses, and oversized error bodies.
  • Bump version to 1.7.9.

Closes #48

Verification

  • dotnet test api/ApplyTrack.slnx --filter "FullyQualifiedName~OpenAiCompatibleLlmClientTests|FullyQualifiedName~LlmConfigTests" --no-restore
  • dotnet build api/ApplyTrack.slnx --no-restore
  • dotnet format api/ApplyTrack.slnx --verify-no-changes --no-restore --include api/ApplyTrack.Api/Llm/OpenAiCompatibleLlmClient.cs api/ApplyTrack.Api.Tests/OpenAiCompatibleLlmClientTests.cs
  • uv run --extra dev pytest -q
  • git diff --check

Full .NET integration tests require the Podman/Testcontainers socket and should run in CI or via ./scripts/test-dotnet-podman.sh on a Podman-enabled host.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Security/stability: cap LLM response bodies before reading into memory

1 participant